Computational Sensors for Global Operations in Vision

The performance of existing machine vision systems still significantly lags that of a biological vision. The two most critical features presently missing from the machine vision are low latency processing and toppdow1z sensory aduptation. This thesis proposes to overcome these two deficiencies by implementing global operations in computational sensors. Computational se.nsors incorporate computation at the level of sensing and can both reduce latency ;md facilitate topdown sensory adaptation. In the context of this thesis the global operations are important because: (1) in perception each decision is a kind of global, or overall, conclusion necessary for the coherent interaction with the environment, and (2) global ope]-ations produce afew quantities for the description of the environment which can be quickly transferred andor processed to produce an appropriatc action for a machine. The main difficulty with implementing global operations comes from the necessity to bring together all or most of the data in the input data set. This work proposes two mechanisms for implementing global operations in computational sensors: ( 1) sensory attention, and (2) intensiry-to-rime processing paradzgm. The sensory attention is based on the premise that salient features within the retinal image represent important global features of the entire image. By selecting a small region of interest around the salient feature for subsequent processing, the sensory attention eliminates extraneous information and allows the processor to handle small amount of data at a time. Thc sensory atlention is used for a VLSI implementation of a trucking compufu thd sensor a computational sensor that attends and tracks a visual stimuli in the retinal image. The intensify-to-time processing puradigm is based on the notion that stronger signals elicit responses before weaker ones allowing a global processor to make decisions based only on a few inputs at a time. The key is that some preliminaq decisions about the retinal image can be mdde as soon as the first responses are received. The intensity-to-time processing paradigm is used for the. VLSI implementation of a sorring compuirtlionnl senror a computational sensor that sorts input stimuli by their intensity as they are being sensed. By implementing the tracking and sorting sensors it is demonstrated that the computational sensor paradigm improves latency and provides top-down sensory feedback for more robust performance in computer vision systems.